"thing that provokes one most," by 1917, from pet (n.1) in the adjectival sense "especially cherished thing" (1826), here in jocular or ironic use with peeve (n.) and perhaps a suggestion of pet (n.2).
grammatical mistakes are his pet peeve
pestilential
pestle
pesto
*pet-
pet
pet peeve
PETA
petal
petard
petcock
*pete-